WHU-Stereo: A Challenging Benchmark for Stereo Matching of High-Resolution Satellite Images

Stereo matching of high-resolution satellite images (HRSI) is still a fundamental but challenging task in the field of photogrammetry and remote sensing. Recently, deep learning (DL) methods, especially convolutional neural networks (CNNs), have demonstrated tremendous potential for stereo matching on public benchmark datasets. However, datasets for stereo matching of satellite images are scarce. To facilitate further research, this paper creates and publishes a challenging dataset, termed WHU-Stereo, for stereo matching DL network training and testing. This dataset is created by using airborne LiDAR point clouds and high-resolution stereo imageries taken from the Chinese GaoFen-7 satellite (GF-7). The WHU-Stereo dataset contains more than 1700 epipolar rectified image pairs, which cover six areas in China and includes various kinds of landscapes. We have assessed the accuracy of ground-truth disparity maps, and it is proved that our dataset achieves comparable precision compared with existing state-of-the-art stereo matching datasets. To verify its feasibility, in experiments, the hand-crafted SGM stereo matching algorithm and recent deep learning networks have been tested on the WHU-Stereo dataset. Experimental results show that deep learning networks can be well trained and achieves higher performance than hand-crafted SGM algorithm, and the dataset has great potential in remote sensing application. The WHU-Stereo dataset can serve as a challenging benchmark for stereo matching of high-resolution satellite images, and performance evaluation of deep learning models. Our dataset is available at https://github.com/Sheng029/WHU-Stere

Efficient Match Pair Retrieval for Large-scale UAV Images via Graph Indexed Global Descriptor

SfM (Structure from Motion) has been extensively used for UAV (Unmanned Aerial Vehicle) image orientation. Its efficiency is directly influenced by feature matching. Although image retrieval has been extensively used for match pair selection, high computational costs are consumed due to a large number of local features and the large size of the used codebook. Thus, this paper proposes an efficient match pair retrieval method and implements an integrated workflow for parallel SfM reconstruction. First, an individual codebook is trained online by considering the redundancy of UAV images and local features, which avoids the ambiguity of training codebooks from other datasets. Second, local features of each image are aggregated into a single high-dimension global descriptor through the VLAD (Vector of Locally Aggregated Descriptors) aggregation by using the trained codebook, which remarkably reduces the number of features and the burden of nearest neighbor searching in image indexing. Third, the global descriptors are indexed via the HNSW (Hierarchical Navigable Small World) based graph structure for the nearest neighbor searching. Match pairs are then retrieved by using an adaptive threshold selection strategy and utilized to create a view graph for divide-and-conquer based parallel SfM reconstruction. Finally, the performance of the proposed solution has been verified using three large-scale UAV datasets. The test results demonstrate that the proposed solution accelerates match pair retrieval with a speedup ratio ranging from 36 to 108 and improves the efficiency of SfM reconstruction with competitive accuracy in both relative and absolute orientation

Abnormal deflection of electrons crossing the boundary of opposite magnetic fields

This paper reports an experiment about abnormal deflection of cathode-ray in odd-symmetric magnetic field. The measurement results show that during cathode-ray passes through odd-symmetric magnetic field, a deflection opposite to Lorentz force occurs at the boundary of magnetic fields. It can be explained by the inertial effect of the electron rotating on its axis in magnetic field, and Lorentz force is similar to the Magnus effect in fluid. In this paper, a mechanical model is used to calculate the force exerted on an electric charge under different conditions, and the Maxwell's equations of electromagnetic field are derived.Comment: 60 pages,51 figure

On-Board GNSS/IMU Assisted Feature Extraction and Matching for Oblique UAV Images

Feature extraction and matching is a crucial task in the fields of computer vision and photogrammetry. Even though wide researches have been reported, some issues are still existing for oblique images. This paper exploits the use of on-board GNSS/IMU (Global Navigation Satellite System/Inertial Measurement Unit) data to achieve efficient and reliable feature extraction and matching for oblique unmanned aerial vehicle (UAV) images. Firstly, rough POS (Positioning and Orientation System) is calculated for each image with cooperation of on-board GNSS/IMU data and camera installation angles, which enables image rectification and footprint calculation. Secondly, two robust strategies, including the geometric rectification and tile strategy, are considered to address the issues caused by perspective deformations and to relieve the side-effects of image down-sampling. According to the results of individual performance evaluation, four combinations of these two strategies are designed and comprehensively compared in BA (Bundle Adjustment) experiments by using a real oblique UAV dataset. The results reported in this paper demonstrate that the solution with the tiling strategy is superior to the other solutions in terms of efficiency, completeness and accuracy. For feature extraction and matching of oblique UAV images, it is proposed to combine the tiling strategy with existing workflows to achieve an efficient and reliable solution

Efficient SfM for Oblique UAV Images: From Match Pair Selection to Geometrical Verification

Accurate orientation is required for the applications of UAV (Unmanned Aerial Vehicle) images. In this study, an integrated Structure from Motion (SfM) solution is proposed, which aims to address three issues to ensure the efficient and reliable orientation of oblique UAV images, including match pair selection for large-volume images with large overlap degree, reliable feature matching of images captured from varying directions, and efficient geometrical verification of initial matches. By using four datasets captured with different oblique imaging systems, the proposed SfM solution is comprehensively compared and analyzed. The results demonstrate that linear computational costs can be achieved in feature extraction and matching; although high decrease ratios occur in image pairs, reliable orientation results are still obtained from both the relative and absolute bundle adjustment (BA) tests when compared with other software packages. For the orientation of oblique UAV images, the proposed method can be an efficient and reliable solution

Boundary-Assisted Learning for Building Extraction from Optical Remote Sensing Imagery

Deep learning methods have been shown to significantly improve the performance of building extraction from optical remote sensing imagery. However, keeping the morphological characteristics, especially the boundaries, is still a challenge that requires further study. In this paper, we propose a novel fully convolutional network (FCN) for accurately extracting buildings, in which a boundary learning task is embedded to help maintain the boundaries of buildings. Specifically, in the training phase, our framework simultaneously learns the extraction of buildings and boundary detection and only outputs extraction results while testing. In addition, we introduce spatial variation fusion (SVF) to establish an association between the two tasks, thus coupling them and making them share the latent semantics and interact with each other. On the other hand, we utilize separable convolution with a larger kernel to enlarge the receptive fields while reducing the number of model parameters and adopt the convolutional block attention module (CBAM) to boost the network. The proposed framework was extensively evaluated on the WHU Building Dataset and the Inria Aerial Image Labeling Dataset. The experiments demonstrate that our method achieves state-of-the-art performance on building extraction. With the assistance of boundary learning, the boundary maintenance of buildings is ameliorated

A Novel Method for High-Voltage Bundle Conductor Reconstruction from Airborne LiDAR Data

The security of high-voltage power transmission corridors is significantly vital to the national economy and daily life. With its rapid development, LiDAR (Light Detection and Ranging) technology has been widely applied in the inspection of transmission lines. As the basis of potential hazard detection, a robust and precise power line model is a necessary requirement for rapid and correct clearance. Thus, this paper proposes a novel method for high-voltage bundle conductor reconstruction, which can precisely reconstruct each sub-conductor. First, points in high-voltage power transmission corridors are detected and classified into four categories; second, for classified power lines, single power line spans are extracted, and bundle conductors are identified by analyzing the single spans’ fitting residuals; and then, each sub-conductor of bundle conductors is extracted by a projected dichotomy method on the XOY and XOZ planes, respectively; finally, a double-RANSAC (random sample consensus)-based algorithm was introduced to reconstruct each power line. The proposed method makes use of the distribution of bundle conductors in high-voltage transmission lines, and our experiments showed that it could preferably reconstruct the real structure of bundle conductors robustly with a high precision better than 0.2 m